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Abstract —In this letter, we consider a problem of recon¬ 
structing an unknown discrete signal taking valnes in a finite 
alphabet from incomplete linear measurements. The difficulty 
of this problem is that the computational complexity of the 
reconstruction is exponential as it is. To overcome this difficulty, 
we extend the idea of compressed sensing, and propose to 
solve the problem by minimizing the sum of weighted absolute 
values. We assume that the probability distribution defined on 
an alphabet is known, and formulate the reconstruction problem 
as linear programming. Examples are shown to illustrate that 
the proposed method is effective. 

Index Terms —Discrete signal reconstruction, sum of absolute 
values, digital signals, compressed sensing, sparse optimization. 

I. Introduction 

Signal reconstruction is a fundamental problem in signal 
processing. Recently, a paradigm called compressed sens¬ 
ing HI, IS, 13 has been proposed for signal reconstruction 
from incomplete measurements. The idea of compressed sens¬ 
ing is to utilize the property of sparsity in the original signal; if 
the original signal is sufficiently sparse, practical algorithms 
such as the basis pursuit a, the orthogonal matching pur¬ 
suit a, etc, may give exact recovery under an assumption on 
the measurement matrix (see e.g. a). 

On the other hand, it is also important to reconstruct 
discrete signals whose elements are generated from a finite 
alphabet with a known probability distribution. This type of 
reconstruction, called discrete signal reconstruction, arises in 
black-and-white or grayscale sparse image reconstruction a> 
0, blind estimation in digital communications a, machine- 
type multi-user communications a, discrete control signal 
design ifTOl . to name a few. 

The difficulty of discrete signal reconstruction is that the re¬ 
construction has a combinatorial nature and the computational 
time becomes exponential. For example, 200-dimensional 
signal reconstruction with two symbols (i.e. binary signal 
reconstruction) needs at worst about 1.5 x 1036 

years with 

a computer of 34 peta FLOPS (see Section |II] below), which 
cannot be executed, obviously. 

To overcome this difficulty, we borrow the idea of com¬ 
pressed sensing based on E optimization as used in the basis 
pursuit 0. Our idea is that if the original discrete signal, say 
X, includes L symbols ri, r 2 ,..., then each vector x — ri 
is sparse. For example, a binary vector x on alphabet {1,-1} 
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includes a number of 1 and —1, and hence both x—1 and x-\-\ 
are sparse. To recover such a discrete signal, we propose to 
minimize the sum of weighted absolute values of the elements 
in X — ri. The weights are determined by the probability 
distribution on the alphabet. The problem is reduced to a 
standard linear programming problem, and effectively solved 
by numerical softwares such as cvx in MATLAB IfTTI . IfT^ . 

For discrete signal reconstruction, there have been re¬ 
searches based on compressed sensing, called integer com¬ 
pressed sensing: 03 has proposed a Bayesian-based method, 
which works only for binary sparse signals (i.e., 0-1 valued 
signals that contain many O’s). 0 considers arbitrary finite 
alphabet that contains 0. More recently, motivated by decision 
feedback equalization, IH proposes to use optimization 
for discrete signal estimation under the assumption of sparsity. 
ifTSl . ifTbl . ifTI also propose methods based on the finiteness of 
the measurement matrix (i.e., the elements of the measurement 
matrix are also in a finite alphabet). As mentioned in ifTSl . 
this type of integer compressed sensing is connected with 
error correcting coding. Compared with these researches, 
the proposed method in this paper considers arbitrary finite 
alphabet that does not necessarily contain 0. 

The remainder of this letter is organized as follows: Section 
mi formulates the problem of discrete signal reconstruction and 
discusses the difficulty of the problem. Section HIH proposes to 
use the sum of weighted absolute values for discrete signal 
reconstruction, and show a sufficient and necessary condition 
for exact recovery by extending the notion of the null space 
property iflTl . Examples of one-dimensional signals and two- 
dimensional images are included in Section |IV] Section [V] 
draws conclusions. 

Notation 

For a vector x = [xi,X 2 , ■ ■ ■ ,a:Ar]^ G we define the 
E and E norms respectively as 

N 

n—1 

where T denotes the transpose. For a vector x and a scalar r, 
we define 

X — r E [xi — r,X2 — r,... ,xn — r]^. 

For a matrix $ £ ker4> is the kernel (or the null 

space) of $, that is, 

ker$ = {x € : ^x = 0}. 
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In is the n-dimensional identity matrix, and is a k- 
dimensional vector whose elements are all 1, that is, 




For two matrices A € and B G A®B denotes 

the Kronecker product, that is. 


AiiB 

A12B 

AimB 

AmiB 

Am2B . 

■ AmnB 




where Aij is the ij-\h element of A. For two real-valued 
vectors x and y of the same size, x < y denotes the element¬ 
wise inequality, that is, x < y means Xi < yi for all i. 


II. Problem Formulation 
A ssume that the original signal x is an A^-dimensional 
vector whose elements are discrete. That is, 

X = [xi,X 2 , ■ ■ . ,a;7v]^ G , 

■T = {ri,r2,... ,rL}, 
where G M and we assume 

ri <r2 < ■ ■ ■ <rL- (1) 

If a symbol, say r, is a complex number, then taking Vj = 
Re(r) and = Im(r) gives a real-valued alphabet, and hence 
the assumption ([TJ is not restrictive. We here assume that the 
values of are known and the probability distribution of them 
is given by 

P(Ti)=Pu t = l,2,...,L, (2) 

where we assume 


Pi > 0, Pi -bp2 H-f Pi = 1- (3) 

Then we consider a linear measurement process modelled 
by 

y = $a;GC^, (4) 


where $ G . Note that we consider a complex-valued 

matrix 4> since $ can be constructed from e.g. a complex¬ 
valued DFT (Discrete Fourier Transform) matrix; see the im¬ 
age processing example in Section |IV] We assume incomplete 
measurements, that is, M < N. The objective here is to 
reconstruct x G from the measurement vector y G 
in dUi. 

First of all, we discuss the uniqueness of the solution 
of the discrete signal reconstruction. We have the following 
proposition: 

Proposition 1: Given $ G the following properties 

are equivalent: 

(A) If <I>a;i = ^X 2 and both a;i and X 2 are in X^, then 

Xi = X2. 

(B) Define the difference set of X as 

X = {ri- Tj :i,j = 


Then 


ker$n = { 0 }. 


(5) 


(C) The matrix $ is injective as a map from X^ to C^. 


Proof: (A)=^>(B): Assume (A) holds. Take any v G 
ker$ n X^. Since v G X^, there exist a:i,a ;2 G X^ such 
that V = Xi — X 2 . Then we have 4>u = — <I>a ;2 = 0 since 

V G ker$. Then from (A), we have a;i = X 2 . It follows that 

V = 0. 

(B) =^>(C): Assume (B) holds. Take any v G X^ and assume 
= 0. Then from (B), we have d = 0. This proves <I> is an 

injective map from X^ to 

(C) =^>(A): Assume that (C) holds. Take any a;i,a :2 G X^ 

such that $011 = $a; 2 . Since X 1 —X 2 G X^ and $ is injective 
on X^, we have Xi — X 2 = 0, or a;i = X 2 . ■ 

Throughout the letter, we assume the uniqueness of the 
solution, that is, the pair {X, $) is chosen to satisfy Q. If the 
uniqueness assumption holds, we can find the exact solution 
in a finite number of steps via an exhaustive computation 
as follows. The set X^ is a finite set, and we can write 
X^ = {xi, X 2 ,..., tC/j}- For each Xi, we compute y^ = ^Xi 
and check if y^ = y. Thanks to the uniqueness assumption, 
we can find the exact solution in a finite time. The problem is 
that the size of X^ is p = , and hence the computational 

complexity is exponential. For example, if L = 2 (two 
symbols) and N = 200, then 

p = 2^°° « 1.7 X 10®°, 

which takes at worst about 1.5 x 10°® years (much longer than 
the lifetime of the universe) by the current fastest computer 
with 34 peta FLOPS. To overcome this, we adopt a relaxation 
technique based on the sum of absolute values in the next 
section. 

III. Solution via Sum of Absolute Values 

We here propose a relaxation method for discrete signal 
reconstruction. By borrowing the idea of compressed sensing, 
we can assume that each vector x — ri, i = 1,2,..., L, is 
sparse (if the probability pi given in (|2]i is not so small), and 
the sparsity is proportional to the probability pi. Hence, we 
consider the following minimization problem: 

L 

minimize F{z) = pi\\z — ri\\i subject to y = ^z, (6) 

where we use the norm for the measure of sparsity as 
in compressed sensing. By the definition of the norm, we 
rewrite the cost function F{z) as 

L N N 

^i^) = 'I2Pi'I2\^n-ri\='^L{Zn), (7) 

i—\ n—1 n—1 

where L[t) is the sum of weighted absolute values 

L 

Lit) = ^p,\t - ri\. 

i=l 

An example of this function is shown in Fig. [T] As is shown 
in this figure, the function L(t) is continuous, convex, and 
piecewise linear. In fact, we have the following proposition: 
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Proposition 2: The function L{t) is continuous and convex 
on R and is a piecewise linear function given by 


1 

i-t + r. 

if f G (—oo, ri] 


Lit) = 

1 (tit X bi, 

if f G iri,ri+i], i = l,2,.. 

.,L-1 

1 

- r, 

if t G [rL,oo). 


where 





r = 

i L i L 

H Pi’ - -^Pj^J + Po^3- 

j=i i=j+i i=i j=i+i 

Proof: Since each function \t — ri\ in L{t) is continuous 
and convex on R, the function L{t), which is the convex 
combination of \t — ri\, i = is also continuous 

and convex on R. 

Suppose t < ri. From the inequality O, we have t — ri < 0 
for i = 1,2,..., L, and hence 

L 

L{t) = - Ti) = -t + r, 

2 = 1 

where we used Q. Next, suppose Vi < t < n+i {i = 
1, 2,..., T — 1). The inequality O gives t — rj >0 for 
j = 1,2,... ,i and t — rj < 0 for j = i + l,i + 2,..., L. 
It follows that 

i L 

Lit) = Ypi^* ~ “ Y Pi^* ~ ’’j) = 

j=l f=i+l 

Finally, if f > r^, then t — ri > 0 for z = 1, 2,..., L due to 
O, and hence 

L 

L{t) = Ypi(^ -ri)=t-r. 

i=l 

■ 

Note that the function L{f) is rewritten as 

Lit) = max {ait + bi}, 

2=0,1,...,L 

where qq = —1, bo = r, = 1, and b^ = —r. It follows 
that the optimization (|6l) is equivalently described as 

minimize ijjd 

eeR" (8) 

subject to y = ^z, Az + b < E6, 


where 6 G R^ is an auxiliary variable, and 

A = In ® a G a = [oq, Oi,..., G 

b= IjvObe R^(^+i\ 6 4 [6o,6i,...,6i]^ € R^+\ 

This is a standard linear programming problem and can be 
efficiently solved by numerical optimization softwares, such 
as cvx in MATLAB HU, ifTH . 

Now, we discuss the validity of the relaxation optimization 
given in © or ®. To see this, we extend the notion of the null 
space property ifTTll in compressed sensing to our problem: 

Definition 1: A matrix $ G is said to satisfy the 

null space property for an alphabet X if 

Fix) < Fix — v), (9) 

for any x G X^ and any v G ker <I> \ {0}. 

Then we have the following theorem: 

Theorem 1: Let $ G . Every x G X^ is uniquely 

recovered from the optimization © with y = <I>a; if and 
only if $ satisfies the null space property for X. 

Proof: (=^): Take any v G ker <I> \ {0} and x G X^. Put 
2 = x — v. Since v G ker $, we have = $(« —at+at) = 0, 
or $a; = $ 2 . This means that 2 is in the feasible set of the 
optimization problem © with y = $a;. Also, we have x z 
since v 0. Now, by assumption, x is the unique solution 
of © with y = $a;, and hence we have Fix ) < Fiz) = 
Fix — v). 

(<^=): Take any x G X^ and 2 G such that x z and 
^x = $ 2 . Put V = x—z. Then we have v G ker cl>\{0}. From 
the null space property for X, we have Fix) < Fix — v) = 
Fiz). It follows that x is the unique solution of ©. ■ 

IV. Examples 

In this section, we show two examples to illustrate the 
effectiveness of the proposed method. 

The first example is one-dimensional signal reconstruction 
with multiple symbols. Let the original signal a; be a 200- 
dimensional vector (i.e. N = 200) and the elements are drawn 
from the following alphabets with probability distributions: 

^-2 ^ {0,1} : P(0) = p, P(l) = l-p, 

A-a 4 {-1, 0,1} : P(0) = p, P(l) = P(-l) = 

A (10) 

A’5 4{-2,-1,0,1,2}:P(0)=p, 

P(-2) = P(-l) = P(l) = P(2) = 

where p G [0,1]. We assume the measurement vector y is a 
100-dimensional vector (i.e. M = 100), and the measurement 
matrix $ is generated such that each element is independently 
drawn from the Gaussian distribution with a mean of 0 and 
a standard deviation of 1 by using a MATLAB command, 
randn (100,200). The original vector x is also generated 
such that each element is independently drawn from the 
distribution ( flOl i with varying parameter p G [0,1]. Pig. |2] 
shows the graphs of the averaged NSR (noise-to-signal ratio) 
jja; - x|| 2 /||a :||2 for A’2,A’3,A’5 in (HOj, where x is the 
reconstructed signal, with 200 trials of random $ and x for 
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Fig. 2. Averaged NSR ||a; — ®|| 2/||®||2 vs probability p for X 2 (top), A '3 
(middle), (bottom), by the proposed (solid) and the basis pursuit (dash) 


each p e [0,1]. If p is large (i.e., p ~ 1), then the original 
vector X is sparse, and we also reconstruct the original signal 
by the basis pursuit 

mii^ ll^lli subject to y = (Pz, 

and then round off the values by the basis pursuit to the nearest 
integer. The error graphs by the basis pursuit are also shown in 
Fig. El For the binary alphabet X 2 = {0,1}, one may exchange 
the roles of 0 and 1 before performing the basis pursuit, and 
the error curve below p = 0.5 is pessimistic. However, such a 
simple strategy cannot be applied to A 3 and A 5 for the basis 
pursuit, while the proposed method works well for small p. 
This is because the basis pursuit does not fully utilize the 
information of the alphabet (i.e., the basis pursuit only uses 
the information of the value 0 through the sparsity). We also 
note that the basis pursuit can be used only when the alphabet 
includes 0 , while the proposed method works as well when 0 is 
not an element of the alphabet. Fig.|2]also implies a conjecture 
that the performance of the proposed method converges that 
of the basis pursuit as the size of the alphabet goes to infinity. 

Next, we see an example from image processing. Let us 
consider a binary (or black-and-white) image shown in Fig. [3 
(left), which is a 37 x 37-pixel binary-valued image. We add 
random Gaussian noise with a mean of 0 and a standard 
deviation of 0.1 to each pixel to obtain a disturbed image as 
shown in Fig. [3 (right). We represent this disturbed image as 
a real-valued matrix X £ Then we apply the discrete 

Fourier transform (DFT) to X to obtain 

X = WXW (11) 

where W is the DFT matrix defined by 

1 ... 1 ■ 




1V = 


1 1 

1 w 


1 UJ 


K-l 



Fig. 3. Original image (left) and disturbed image by random noise (right) 



Fig. 4. Reconstnacted images by the basis pursuit (left) and by the proposed 
method (right) 


where K = 37 and w = exp(—j27r/iT). The relation can be 
equivalently represented by 

vec(V) = (IL (g) W)vec{X) G 

We then randomly down-sample the vector vec(X) to obtain a 
half-sized vector y G C®®®. The measurement matrix $ is then 
a 685 X 1369 matrix generated by randomly down-sampling 
row vectors from W 0 W. Fig. |4] shows the reconstructed 
images by the basis pursuit with rounding off (left) and by 
the proposed method (right). For the proposed method, we 
assumed P(0) = P(l) = 1/2. The results clearly show the 
effectiveness of our method also for image reconstruction. 

V. Conclusion 

In this letter, we have proposed a reconstruction method for 
discrete signals based on the sum of absolute values (or the 
weighted norm). The reconstruction algorithm is described 
as linear programming, which can be solved effectively by 
numerical optimization softwares. Examples have been shown 
that the proposed method is much more effective than the 
basis pursuit which only uses the information of sparsity. 
Future work includes an accessible condition that ensures the 
null space property, as the restricted isometry property in 
compressed sensing. 
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